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Cognitive computation with autonomously active neural 
networks: an emerging field 
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Abstract The human brain is autonomously active. 
To understand the functional role of this self-sustained 
neural activity, and its interplay with the sensory data 
input stream, is an important question in cognitive sys- 
tem research and we review here the present state of 
theoretical modelling. 

This review will start with a brief overview of the 
experimental efforts, together with a discussion of tran- 
sient vs. self-sustained neural activity in the framework 
of reservoir computing. The main emphasis will be then 
on two paradigmal neural network architectures show- 
. ing continuously ongoing transient-state dynamics: sad- 
dle point networks and networks of attr actor relics. 

Self-active neural networks are confronted with two 
■ seemingly contrasting demands: a stable internal dy- 
namical state and sensitivity to incoming stimuli. We 
show, that this dilemma can be solved by networks of 
attractor relics based on competitive neural dynamics, 
where the attractor relics compete on one side with each 
other for transient dominance, and on the other side 
with the dynamical influence of the input signals. 

Unsupervised and local Hebbian-style online learn- 
ing then allows the system to build up correlations be- 
tween the internal dynamical transient states and the 
sensory input stream. An emergent cognitive capability 
results from this set-up. The system performs online, 
and on its own, a non-linear independent component 
analysis of the sensory data stream, all the time be- 
ing continuously and autonomously active. This process 
maps the independent components of the sensory input 



Claudius Gros 

Institute of Theoretical Physics 
J.W. Goethe University Frankfurt 
60054 Frankfurt/Main, Germany 



onto the attractor relics, which acquire in this way a 
semantic meaning. 

Keywords recurrent neural networks • autonomous 
neural dynamics • transient state dynamics • emergent 
cognitive capabilities 



1 INTRODUCTION 

The brain has a highly developed and complex self- 
generated dynamical neural activity, and this fact raises 
a series of interesting issues. Does this self-sustained 
neural dynamics, its eigendynamics, have a central func- 
tional role, organizing overall cognitive computational 
activities? Or does this ongoing autonomous activity 
just serve as a kind of background with secondary com- 
putational task, like non-linear signal amplification or 
time encoding of neural codes? 

The answer to this question is important not only to 
system neurobiology, but also for research in the field of 
cognitive computation in general. We will review here 
approaches based on the notion that the autonomous 
neural dynamics has a central regulating role for cog- 
nitive information processing. We will then argue, that 
this line of research constitutes an emerging field in 
both computational neuroscience and cognitive system 
research. 

Some preliminaries, before we start. This is a mostly 
non-technical review with emphasis on content, an ex- 
haustive and complete discussion of the published work 
on the subject is not the objective here. Centrally im- 
portant equations will be given and explained, but for 
the numerical values of the parameters involved, and 
for the details of the simulation set-ups, we will refer 
to the literature. The discussion will be given gener- 
ally from the perspective of cognitive system theory, viz 
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bearing in mind the overall requirements of prospective 
complete cognitive systems, akin to ones of real- world 
living animals [TJ[2]. 



1.1 Autonomous brain dynamics 

On the experimental side, the study of self-induced or 
autonomous neural activity in the brain has seen sev- 
eral developments in recent years, especially by fMRI 
studies [3], and we will start by discussing some key 
issues arising in this respect. 

The vast majority of experiments in cognitive neu- 
roscience study the evoked neural response to certain 
artificial or natural sensory stimuli, often involving a 
given task which has been trained previously. It has 
been realized early on, that the neural response shows 
strong trial-to-trial variation, which is often as large as 
the response itself. This variability in the response to 
identical stimuli is a consequence of the ongoing internal 
neural activities (for a discussion see [4J). Experimen- 
tally one has typically no control over the details of the 
internal neural state and it is custom to consider it as a 
source of noise, averaging it out by performing identi- 
cal experiments many times over. It is on the other side 
well known that the majority of energy consumption 
of the brain is spent on internal processes [5], indicat- 
ing that the ongoing and self-sustained brain dynamics 
has an important functional role. Two possibilities are 
currently discussed: 

(A) - The internal neural activity could be in essence 
a random process with secondary functional roles, such 
as non-linear signal amplification or reservoir comput- 
ing for the spatiotemporal encoding of neural signals 
(for a theory review see [6]). 

(B) - The internal neural activity could represent 
the core of the cognitive information processing, be- 
ing modulated by sensory stimuli, but not directly and 
forcefully driven by the input signals. Indications for 
this scenario arise, e.g., from studies of the visual infor- 
mation processing [7J and of the attention system [8]. 

The overall brain dynamics is still poorly under- 
stood and both possibilities (A) and (B) are likely to 
be relevant functionally in different areas. In this re- 
view we will focus on the ramifications resulting from 
the second hypothesis. There are indications, in this re- 
gard, that distinct classes of internal states generated 
autonomously correspond to dynamical switching cor- 
tical states, and that the time series of the spontaneous 
neural activity patterns is not random but determined 
by the degree of mutual relations [9] . Additionally, these 
spontaneous cortical state may be semantic in nature, 
having a close relation to states evoked by sensory stim- 



uli [10] and to neural activity patterns induced via tha- 
lamic stimulation [11]. A second characteristics recur- 
rently found in experimental studies is the organization 
of the spontaneously active states into spatially anticor- 
related networks [12J, being transiently stable in time, 
in terms of firing rates, with rapid switching between 
subsequent states [T3] . 

These results indicate that certain aspects of the 
time evolution of the self-sustained neural activity in 
the brain have the form of transient state dynamics, 
which we will discuss in detail in Sect. El together with 
a high associative relation between subsequent states of 
mind. This form of spontaneous cognitive process has 
been termed 'associative thought process' [T4] , 

It is currently under debate which aspects of the in- 
trinsic brain dynamics is related to consciousness. The 
global organization of neural activity in anticorrelated 
and transiently stable states has been suggested, on one 
side, to be of relevance also for the neural foundations of 
consciousness [EJUS], viz the 'observing self [17]. The 
persistent default-mode network (for a critical perspec- 
tive see [E]), viz the network of brain areas active in the 
absence of explicit stimuli processing and task perfor- 
mance, has been found, on the other side, to be active 
also under anesthetization [19] and light sedation [20] . 
It is interesting to note, in this context, that certain 
aspects of the default resting mode can be influenced 
by meditational practices [2T] , 

1.2 Reservoir computing 

The term 'neural transients' characterizes evoked pe- 
riods of neural activities, remaining transiently stable 
after the disappearance of the primary stimulating sig- 
nal. In the prolonged absence of stimuli, neural archi- 
tectures based on neural transients relax back to the 
quiescent default state. Network setups based on neu- 
ral transients therefore occupy a role functionally in 
between pure stimulus-response architectures and sys- 
tems exhibiting continuously ongoing autonomous neu- 
ral activity. An important class of neural architectures 
based on neural transients are neural reservoirs, which 
we discuss now briefly. 

A recurrent neural net is termed a reservoir, if it is 
not involved in the primary cognitive information pro- 
cessing, having a supporting role. A typical architecture 
is illustrated in Fig. [H The reservoir is a randomly con- 
nected network of artificial neurons which generally has 
only a transiently stable activity in the absence of in- 
puts, viz the reservoir has a short-term memory. 

In the standard mode of operation an input signal 
stimulates the network, giving raise to complex spa- 
tiotemporal reservoir activities. Normally, there is no 




Fig. 1 Operating principle of reservoir dynamics. The reservoir is activated by the input signal, mapping it in time and space to a 
higher-dimensional space. The reservoir activity provides then the basis for linear and supervised training of the output units, typically 
for time prediction tasks (figure courtesy H. Jaeger). 



internal learning inside the reservoir, the intra-reservoir 
synaptic strengths are considered fixed. Time predic- 
tion is the standard application range for reservoir com- 
puting. For this purpose the reservoir is connected to an 
output layer and the activities of the output neurons are 
compared to a teaching signal. With supervised learn- 
ing, either online or off-line, the links leading from the 
reservoir to the output then acquire a suitable synaptic 
plasticity. 

There are two basic formulations of reservoir com- 
puting. The 'echo-state' approach using discrete-time 
rate-encoding neurons [22,23j, and the 'liquid state ma- 
chine' using continuous-time spiking neurons [241 125]. In 
both cases the dimensionality of the input signal, con- 
sisting normally of just a single line, is small relative 
to the size of the reservoir, which may contain up to 
a few hundred neurons. Many nonlinear signal trans- 
formations are therefore performed by the reservoir in 
parallel and the subsequent perceptron-like output neu- 
rons may solve complex tasks via efficient linear learn- 
ing rules. 

Neural reservoirs are possible candidates for local 
cortical networks like microcolumns. The bare-bones 
reservoir network is not self-active, but feedback links 
from the output to the reservoir may stabilize ongoing 
dynamical activity [26]. In any case, reservoir nets are 
examples of network architectures of type (A), as de- 
fined in the previous section. The task of the reservoir, 
non-linear signal transformation, is performed automat- 
ically and has no semantic content. All information is 
stored in the efferent synaptic links. 



There is an interesting similarity, on a functional 
level, of reservoir computing with the notion of a 'global 
workspace' [27,28j. The global workspace has been pro- 
posed as a global distributed computational cortical 
reservoir, interacting with a multitude of peripheral lo- 
cal networks involving tasks like sensory preprocessing 
or motor output. The global workspace has also been 
postulated to have a central mediating role for con- 
scious processes [27(128]. representing the dominating 
hub nodes of a large-scale, small- world cortical network 

ESI- 

2 TRANSIENT STATE DYNAMICS 

A central question in neuroscience regards the neural 
code, that is the way information is transmitted and 
encoded (see [3Q|l3Tj for reviews). Keeping in mind that 
there is probably no pure information transmission in 
the brain, as this would be a waste of resources, that 
information is also processed when transmitted, one 
may then distinguish two issues regarding the encod- 
ing problem. 

On one hand there is the question on how sensory 
signals are reflected, on relative short timescales, in sub- 
sequent neural activities. Available neural degrees of 
freedom for this type of short-time encoding are the 
average firing rates (rate encoding), transient bursts of 
spikes and the temporal sequence of spikes (temporal 
encoding). In addition, the response of either individ- 
ual neurons may be important, or the response of local 
ensembles [3T ] l32]. 
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Fig. 2 Two examples of transient state dynamics. Left: Bump-like phases of activities typically result from trajectories passing close 
to saddle points. Right: Transient states with pronounced plateaus are typical for multi-winners-take-all set-ups in the context of 
networks with attractor relics. 



The subsequent sensory signal processing, on time- 
scales typically exceeding 25- 100ms, may, on the other 
hand, involve neural dynamics in terms of transiently 
stable activity patterns, as discussed earlier in Sect. II. ll 
In Fig. [2] two types of model transient state activities 
are illustrated. Alternating subsets of neurons are either 
active, to various degrees, or essentially silent, resulting 
in well characterized transient states having a certain 
degree of discreteness. This discreteness should be re- 
flected, on a higher level, on the properties of the cor- 
responding cognitive processes. Of interest in this con- 
text is therefore the ongoing discussion, whether visual 
perception is continuous or discrete in the time domain 
[33,34J, on timescales of the order of about 100ms, with 
the discrete component of perception possibly related 
to object recognition [35]. Transient state dynamics in 
the brain may therefore be related to semantic recog- 
nition, a connection also found in models for transient 
state dynamics based on competitive neural dynamics. 
In the following we will examine the occurrence and the 
semantic content of autonomous transient state dynam- 
ics in several proposed cognitive architectures. 



2.1 Saddle point networks and winnerless competition 

The concept of saddle point networks is based on the 
premises, (a) that the internal ongoing autonomous dy- 
namics organizes the cognitive computation and (b) 
that the cognitive behavior is reproducible and deter- 
ministic in identical environments [36]. As we will dis- 
cuss in the next section, the first assumption is shared 
with attractor relic networks, while the second is not. 

Technically, one considers a dynamical system, viz 
a set of N first-order differential equations and the set 
of the respective saddle points, compare Fig. 03 The 
precondition is now that every saddle point has only a 
single unstable direction and (TV — 1) stable directions. 
Any trajectory approaching the saddle point will then 
leave it with high probability close to the unique un- 
stable separatrix and the system therefore has a unique 
limiting cycle attractor. This limiting cycle does not 



need to be a global attractor, but normally has a large 
basin of attraction. During one passage most, if not all, 
saddle points are visited one after the other, giving raise 
to a transient state dynamics illustrated in Fig. [2j with 
the trajectory slowing down close to a saddle point. 

Another condition for this concept to function is the 
formation of a heteroclinic cycle, which is of a set in 
phase space invariant under time evolution [37] . Imply- 
ing, as illustrated in Fig. 03 that the unstable separatrix 
of a given saddle point needs to end up as a stable sep- 
aratrix of another saddle point. Such a behavior occurs 
usually only when the underlying differential equations 
are invariant under certain symmetry operations, like 
the exchange of variables [37]. For any practical ap- 
plication, these symmetries need to be broken and the 
limiting cycle will vanish together with the heteroclinic 
sequence. It can however be restored in form of a hetero- 
clinic channel, if the strength of the symmetry-breaking 
is not too strong, by adding a stochastic component to 
the dynamics. With noise, a trajectory loitering around 
a saddle point can explore a finite region of phase space 
close to the saddle point until it finds the unstable direc- 
tion. Once the trajectory has found stochastically the 
unstable direction, it will leave the saddle point quickly 
along this direction in phase space and a heteroclinic 
channel is restored functionally. Cognitive computation 
on the backbone of saddle point networks is therefore 
essentially based on an appropriate noise level. 

Cognitive computation with saddle point networks 
has been termed 'winnerless competition' in the context 
of time encoding of natural stimuli [38] and applied to 
the decision making problem. In the later case interac- 
tion with the environment may generate a second un- 
stable direction at the saddle points and decision taking 
corresponds to the choice of unstable separatrix taken 
by the trajectory [36] . 



2.2 Attractor relic networks and slow variables 

A trivial form of self-sustained neural activity occurs in 
attractor networks [39]. Starting with any given initial 
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Fig. 3 Two scenarios giving rise to transient state dynamics. Left: A network of saddle points (filled circles) connected via heteroclines 
(long dashed lines) with a sample trajectory (short-dashed line). The dynamics slows down close to a saddle point. Right: An attractor 
network (shaded plane) is embedded in a higher dimensional space via additional reservoir variables {y>j(t)}, leading to a destruction 
of the original fixpoints (filled circles), which are turned into attractor relics. The dynamics {xi(t),(pj(t)} (short-dashed line) slows 
down in the vicinity of an attractor relic. 



state the network state will move to the next attractor 
and stay there, with all neurons having a varying de- 
gree of constant firing rates, the very reason attractor 
nets have been widely discussed as prototypes for the 
neural memory [40]. As such, an attractor network is 
useless for a cognitive system, as it needs outside help, 
or stimuli from other parts of the system, to leave the 
current attractor. 

There is a general strategy which transforms an at- 
tractor network into one exhibiting transient state dy- 
namics, with the transient neural states corresponding 
to the fixpoints of the original attractor network. This 
procedure is applicable to a wide range of attractor net- 
works and consists in expanding the phase space by 
introducing additional local variables akin to local ac- 
tivity reservoirs [T4j . 

To be concrete, let us denote with {xi} the set of 
dynamical variables of the attractor network, as illus- 
trated in Fig. El and by {tfj} the additional reservoir 
variables. We assume that the reservoirs are depleted / 
filled when the neuron is active / inactive, 

. /jtN f < active neuron j ... 
^ J [ > inactive neuron j 

Together with a suitable coupling of the reservoir vari- 
ables {tpj} to the neural activities {xi\ one can easily 
achieve that the fixpoints of the attractor networks be- 
come unstable, viz that they are destroyed, turning into 
attractor ruins or attractor relics. 

This situation is illustrated in Fig. [3l In the ex- 
panded phase space {x^, cpj} there are no fixpoints left. 
It is not the case that the attr actors would just acquire 
additional unstable directions, upon enlargement of the 
phase space, turning them into saddle points. Instead, 
the enlargement of the phase space destroys the origi- 
nal attr actors completely. The trajectories will however 
still slow down considerably close to the attractor ru- 
ins, as illustrated in Fig. [21 if the reservoirs are slow 
variables, changing only relatively slowly with respect 
to the typical time constants of the original attractor 



network. In this case the time constant entering the 
time evolution of the reservoir, Eq. (pQ), is large. In the 
limit — > oo the reservoir becomes static and the dy- 
namics is reduced to the one of the original attractor 
network. 

The dynamics exhibited by attractor relic networks 
is related to the notion of chaotic itinerancy [41] , which 
is characterized by trajectories wandering around chaot- 
ically in phase space, having intermittent transient peri- 
ods of stability close to attractor ruins. Here we consider 
the case of attractor relics arising from destroyed point 
attractors. In the general case one may also consider, 
e.g., limit cycles or strange attractors. 

The coupling to slow variables outlined here is a 
standard procedure for controlling dynamical systems 
[1], and has been employed in various fashions for the 
generation and stabilization of transient state dynam- 
ics. One possibility is the use of dynamical thresholds 
for discrete-time rate-encoding neural nets [42] . In this 
case one considers as a slow variable the sliding-time av- 
eraged activity of a neuron and the threshold of a neu- 
ron is increased / decreased whenever the neuron is ac- 
tive / inactive for a prolonged period. Another approach 
is to add slow components to all synaptic weights for 
the generation of an externally provided temporal se- 
quence of neural patterns [4!| . In the following we will 
outline in some detail an approach for the generation 
of transient state dynamics which takes an unbiased 
clique encoding neural net as its starting point [44] . 
with the clique encoding network being a dense and 
homogeneous associative network (dHan). 

3 COMPETITIVE NEURAL DYNAMICS 

Transient state dynamics is intrinsically competitive in 
nature. When the current transient attractor becomes 
unstable, the subsequent transient state is selected via 
a competitive process. Transient-state dynamics is a 
form of 'multi- winners-take- all' process, with the win- 
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Fig. 4 Multi-winners-take-all transient state dynamics and neural competition. The right graph is a blow-up of the first transition 
in the left graph. During the transition many neurons compete with each other for becoming a member of the next winning coalition. 
The duration of the transient-state plateaus is given by the relaxation time of the slow variables, see Eq. {T}. 



ning coalition of dynamical variables suppressing all 
other competing activities [45,46]. Competitive proces- 
ses resulting in quasi-stationary states with intermit- 
tent burst of changes are widespread, occurring in many 
spheres of the natural or the social sciences. In the con- 
text of Darwinian evolution, to give an example, this 
type of dynamics has been termed 'punctuated equilib- 
rium' [lj. In the context of research on the neural corre- 
lates of consciousness, these transiently stable states in 
form of winning coalitions of competing neural ensem- 
bles have been proposed as essential building blocks for 
human states of the mind [47] [48] . 

The competitive nature of transient state dynam- 
ics is illustrated in Fig. [4j where a representative result 
of a simulation for a dHan net is presented. During the 
transition from one winning coalition to the subsequent, 
many neurons try to become members of the next win- 
ning coalition, which in the end is determined by the 
network geometry, the synaptic strengths and the cur- 
rent reservoir levels of the participating neurons. 

The transition periods from one transient state to 
the next are periods of increased dynamical sensibil- 
ity. When coupling the network to sensory inputs, the 
input signal may tilt the balance in this competition 
for the next winning coalition, modulating in this way 
the ongoing internal dynamical activity. Transient state 
dynamics therefore opens a natural pathway for imple- 
menting neural architectures for which, as discussed in 
the introduction, the eigendynamics is modulated, but 
not driven, by the sensory data input stream. A con- 
crete example of how to implement this procedure will 
be discussed in Sect. HI 

3.1 Clique encoding 

Only a small fraction of all neurons are active at any 
time in the brain in general, and in areas important for 
the memory consolidation in particular [49]. For vari- 



ous reasons, like the optimization of energy consump- 
tion and the maximization of computational capabili- 
ties [50 , sparse coding is an ubiquitous and powerful 
coding strategy [45]. Sparse coding may be realized in 
two ways, either by small non-overlapping neural en- 
sembles, as in the single- winner-take-all architecture, or 
by overlapping neural ensembles. The latter pathway 
draws support from both theory considerations [49J, 
and from experimental findings. 

Experimentally, several studies of the Hippocampus 
indicate that overlapping neural ensembles constitute 
important building blocks for the real-time encoding of 
episodic experiences and representations [5T|l52] . These 
overlapping representations are not random superposi- 
tions but associatively connected. A hippocampal neu- 
ron could response, e.g., to various pictures of female 
faces, but these pictures would tend to be semantically 
connected, e.g. they could be the pictures of actresses 
from the same TV series [53 . It is therefore likely that 
the memory encoding overlapping representations form 
an associative network, a conjecture that is also consis- 
tent with studies of free associations [54|l55]. 

There are various ways to implement overlapping 
neural encoding with neural nets. Here we discuss the 
case of clique encoding. The term clique stems from 
graph theory and denotes, just as a clique of friends, a 
subgraph where (a) every member of the clique is con- 
nected with all other members of the clique and where 
(b) all other vertices of the graph are not connected to 
each member of the clique. In Fig. [5] a small graph is 
given together with all of its cliques. 

Also shown in Fig. [5] are the associative interconnec- 
tions between the cliques. One may view the resulting 
graph, with the cliques as vertices and with the inter- 
clique associative connections as edg es, as a higher-level 
representation of an implicit hierarchical object defini- 
tion [56J. The clique (4,5,9) in the original graph in 
Fig. [5] corresponds to a primary object and the meta- 
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Fig. 5 Clique encoding. On the left a 9-site network. On the right the 5 cliques contained in the left-side network are given. The 
cliques are connected via edges whenever they share one or more sites and can therefore be considered as the constituent vertices of a 
met a network of cliques. 



clique [(4,5,9)-(2,4,6,7)-(4,5,6,8)] in the graph of the 
cliques would in this interpretation encode a meta ob- 
ject, composed of the primary objects (4,5,9), (2,4,6,7) 
and (4,5,6,8). This intrinsic possibility of hierarchical 
object definitions when using clique encoding has how- 
ever not yet be explored in simulations and may be of 
interest for future studies. 

Cliques can be highly overlapping and there can be a 
very large number of cliques in any given graph [57,65j. 
We will construct now a neural net where the cliques of 
the network are the attractors. It is a homogeneously 
random and dense associative network (dHan), where 
the associative relations between cliques are given by 
the number of common vertices. Starting from this at- 
tractor network we will introduce slow variables, as dis- 
cussed in Sect. 12. 2\ in terms of local reservoirs. The net- 
work will then show spontaneously generated transient 
state dynamics, with the neural cliques as the attr actor 
ruins. In a second step we will couple the dHan net to 
sensory stimuli and study the interplay between the in- 
ternal autonomous dynamical activity and the data in- 
put stream. We will find that the cliques acquire seman- 
tic content in this way, being mapped autonomously to 
the statistically independent patterns of the data input 
stream. 



3.2 Attr actor network 

The starting point of our considerations is the under- 
lying attractor network, for which we employ a con- 
tinuous time formulation, with rate encoding neurons, 
characterized by normalized activity levels X{ G [0,1]. 
The growth rates ri govern the respective time devel- 
opments, 

Xi)ri (u > 0) 

Xi Ti (Ti < 0) 

When n > 0, the respective neural activity X{ increases, 
approaching rapidly the upper bound; when n < 0, it 



decays to zero. We split the rates into three contribu- 
tions: 



(2) 



(3) 

An internal positive contribution > 0, an internal 
negative contribution r\ ^ < 0, and the influence of 
external stimuli, r\ ext ^ . We will discuss the influence of 
non-trivial external stimuli in Sect. 0J for the moment 
we consider r\ ext ^ = 0. The division into an exciting and 
a depressing contribution in Eq. (j3j) reflects on one side 
the well known asymmetry between excitatory neurons 
and inhibitory interneurons in the brain [58] and is on 
the other side essential for clique encoding. The r- + ^ 
are determined via 

= fw(<Pi)^2mjXj (4) 

3 

by the influence of the excitatory synaptic weights, > 
0. The function f w ((f) entering Eq. (|4j) couples the dy- 
namics of the neurons locally to the slow variables ipi. 
We will examine the reservoir function f w ((p) in the 
next Section. For the time being we set f w (ip) = 1, 
the primary neural dynamics is then decoupled from 
the reservoir dynamics and we will retain the starting 
attractor network. The r\ ^ < are given by 



,(-) 



tanh 

Zij Xj 

fzifPi 



(5) 



where the < are the inhibitory synaptic weights 
and where z > sets the scale for the inhibition. Eq. 
([5]) leads to a normalization — z < r\ ^ < 0. We post- 
pone the discussion of the reservoir function f z (^p) and 
consider for the time being f z (^>) = 1. 

Clique encoding corresponds to a multi- winners-take- 
all formulation. An inhibitory background is therefore 
necessary. The dHan architecture contains hence an in- 
hibitory link z^ whenever there is no excitatory link 

-z (wij 0) 
{wij > 0) ' 



(6) 
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viz the excitatory links are shunting the inhibitory syn- 
apses. This inhibitory background is implicitly present 
for the 9-site network shown in Fig. [5] The edges of the 
network shown in Fig. correspond to excitatory links 
Wij > 0. All pairs of sites not connected by an edge in 
Fig. [5] inhibit each other via Zij < 0. 

The formulation of the attractor network with clique 
encoding is such complete [44] . All members of a given 
clique excite each other via intra-clique w^ > 0. Neu- 
rons which are not members of the current active clique 
are suppressed by at least one inhibitory link < 0. 
This suppression ~ (— z), compare Eq. ([5]), dominates 
the residual positive signal the out-of-clique neuron may 
receive, whenever z is large enough. 

An interesting feature of the dHan architecture is 
the absence of a bias in Eq. ([2]) . There is no self excita- 
tion or suppression, wu = zu =0. The dynamics of an 
individual neuron is exclusively driven by the influence 
of the network, it has no preferred firing state. This 
feature would correspond for real- world neurons to the 
existence of a background of afferent activities with a 
level close to the firing threshold. 

Next we note, that the separation of scales z ^> 
Wij implies that Hebbian-type modification of the in- 
hibitory links z^ would be meaningless, small changes 
of a relatively large quantity will not lead to a substan- 
tial effect. Hebbian learning in the dHan architecture 
is therefore operational only for the excitatory links 
Wij , in accordance to the general assumption that most 
learning taking place in the brain involves synapses 
interconnecting neurons and not interneurons [58]. In 
Sect. [4] we will consider the synaptic plasticity of links 
afferent to the dHan layer. Unsupervised and local Hebbian- 
style learning can however be implemented easily for 
the intra-dHan excitatory links w^ for unsupervised 
and homeostatic calibration of the excitatory links [44] . 
It is however not essential for the occurrence and for 
the stabilization of transient state dynamics, our focus 
here. 



3.3 Reservoir activity 

We consider normalized slow variables cpi G [0, 1], with 
the time evolution 

. ( . av If —ifi active neuron i , , 

WW = r ^ + jr | (1 _ ^ inactive neuron i > ( 7 ) 

where a neuron is active / inactive whenever its activity 
level Xi is close to unity / zero. The cpi behave function- 
ally as reservoirs, being depleted / refilled for active / 
inactive neurons. The term r® v . on the RHS of Eq. Q 
is not essential for the establishment of transient state 



dynamics, but opens an interesting alternative interpre- 
tation for the slow variables. r® v . vanishes for inactive 
neurons and takes the value 

Clactivei = ^ E fa-Vi) (8) 
^ active j 

for active neurons. The reservoir levels {(ft} of all ac- 
tive neurons are drawn together consequently. All mem- 
bers of the currently active winning coalition have then 
similar reservoir levels after a short time, on the order 
of T£ v . This is a behavior similar to what one would 
expect for groups of spiking neurons forming winning 
coalitions via synchronization of their spiking times. For 
each neuron of the winning coalitions one could define 
a degree of synchronization, given by the extent this 
neuron contributes to the overall synchronization. Ini- 
tially, this degree of synchronization would have a dif- 
ferent value for each participating neuron. On a certain 
timescale, denoted here by X^, the spiking times would 
then get drawn together, synchronized, and all members 
of the winning coalition of active neurons would then 
participate to a similar degree in the synchronized fir- 
ing. The firing of the winning coalition would however 
not remain coherent forever. Internal noise and exter- 
nal influences would lead to a desynchronization on a 
somewhat longer time scale ^> T£ v . When desyn- 
chronized, the winning coalition would loose stability, 
giving way to a new winning coalition. In this interpre- 
tation the reservoirs allow for a "poor man's" imple- 
mentation of self organized dynamical synchronization 
of neural ensembles, a prerequisite for the temporal cod- 
ing hypothesis of neural object definition [59^ 160]. 

Finally we need to specify the reservoir coupling 
functions f w (ip) and f z (ip) entering Eqs. ([4]) and ([5]). 
They have sigmoidal form with 

and a straightforward interpretation: It is harder to ex- 
cite a neuron with depleted reservoir, compare Eq. ([4]), 
and a neuron with a low reservoir level has less power to 
suppress other neurons, see Eq. ([5]). Reservoir functions 
obeying the relation ([9]) therefore lead in a quite natural 
way to transient state dynamics. On a short time scale 
the system relaxes towards the next attractor ruin in 
the form of a neural clique. Their reservoirs then slowly 
decrease and when depleted they can neither continue 
to mutually excite each other, nor can they suppress the 
activity of out-of-clique neurons anymore. At this point, 
the winning coalition becomes unstable and a new win- 
ning coalition is selected via a competitive process, as 
illustrated in Fig. [U 

Any finite < oo leads to the destruction of the 
fixpoints of the original attractor network, which is thus 
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(0,1,2) (2,3,4) (4,5,6) (6,7) (7,8,9,10) 




Fig. 6 A dHan layer (top) with neural activities {xi} and cliques 
(0,1,2), ... receives sensory signals via the input layer (middle) in 
the form of certain input patterns (bottom). 



turned into an attractor relic network. The sequence of 
winning coalitions, given by the cliques of the network, 
is however not random. Subsequent active cliques are 
associatively connected. The clique (1,9) of the 9-site 
network shown in Fig. [5j to give an example, could 
be followed by either (4,5,9) or by (1,2,3), since they 
share common sites. The competition between these 
two cliques will be decided by the strengths of the ex- 
citatory links and by the history of previous winning 
coalitions. If one of the two cliques had been activated 
recently, the constituent sites will still have a depressed 
reservoir and resist a renewed reactivation. 

The finite state dynamics of the dHan architecture 
is robust. For the isolated network, we will discuss the 
coupling to sensory input in the next section, the dy- 
namics is relaxational and dissipative [44J. The system 
relaxes to the next attractor relic and the reservoirs are 
relaxing either to zero or to unity, depending on the 
respective neural activity levels. For a network with a 
finite number of sites, the long-time state will be a long 
limiting cycle of transient states. 

The simulation results shown in Fig. [4] are for a set 
of parameters resulting in quite narrow transitions and 
long plateaus [55] , The formulation presented here al- 
lows for the modelling of the shape of the plateaus and 
of other characteristics of the transient state dynamics. 
A smaller would result in shorter plateaus, a longer 
T£ v in longer transition times. One can, in addition, ad- 
just the shape of the reservoir functions and details of 
Eqs. (H|) and ([5]) in order to tune the overall competition 
for the next winning coalition. The dHan architecture 
providing therefore a robust framework for the gener- 
ation of transient state dynamics, offering at the same 
time ample flexibility and room for fine tuning, paving 
the way for a range of different applications. 



4 INFLUENCE OF EXTERNAL STIMULI 

The transient state dynamics generated by the dHan 
architecture is dynamically robust. The dHan dynamics 
has at the same time windows of increased sensibility 
to outside influences during the transition periods from 
one transient state to the subsequent, as shown in Fig. 
[H These transition periods are phases of active inter- 
neural competition, reacting sensibly to the influence of 
afferent signals. 

We couple the input signals via an appropriate input 
layer, as illustrated in Fig. El denoting by yi G [0, 1] 
the time dependent input signals, which we will take 
as black-and-white or grey-scaled patterns. We denote 
by Vij the afferent links to the dHan layer, with the 
external contribution to the dHan-layer growth rates, 
compare Eq. (|3]), given by 

(ext) f neuron i active and Ar* < _ N 
A -\ Ari > ( 10 ) 

An = ^v^yj . (n) 

3 

The rationale behind this formulation is the following. 
The role of the input signal is not to destabilize the 
current winning coalition, the afferent signal is there- 
fore shunted off in this case, Eq. ([TO]) . The input signal 
should influence the competition for the next winning 
coalition, modulating but not driving directly the dHan 
dynamics. This rational is realized by the above formu- 
lation. Inactive neurons will receive a bias Ari from 
the input layer which increases / decreases its chance 
of joining the next winning coalition for An > / 

An < o. 



4.1 Novelty signal 

A cognitive system with a non-trivial and self-sustained 
internal neural activity has to decide how and when cor- 
relations with the sensory data input stream are gener- 
ated via correlations encoded in the respective synaptic 
plasticities. This is clearly a central issue, since the in- 
put data stream constitutes the only source for seman- 
tic content for a cognitive system. 

It makes clearly no sense if the afferent links to the 
dHan layer, viz the links leading from the input to the 
internal network supporting a self-sustained dynamical 
activity, would be modified continuously via Hebbian- 
type rules, since the two processes, the internal and 
the environmental dynamics, are per se unrelated. It 
makes however sense to build up correlation whenever 
the input has an influence on the internal activity, mod- 
ulating the ongoing associative thought process. From 
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the perspective of the cognitive system such a modula- 
tion of the internal dynamics by environmental stimuli 
corresponds to something novel and unexpected hap- 
pening. Novelty detection is therefore vital for neural 
networks with a non-trival eigendynamics processing 
sensory data. The importance of novelty detection for 
human cognition has been acknowledged indeed since 
long [6T1I62], and a possible role of Dopamine, tradition- 
ally associated with reinforcement reward transmission 
[66J, for the signalling of novelty has been suggested 
recently [67] . 

The influence of modulating and of not modulating 
sensory signals is illustrated in Fig. [71 where simulation 
results for a dHan layer containing seven neurons cou- 
pled to an intermittent input signal are presented. The 
signal is not able to deactivate a currently stable win- 
ning coalition, compare Eq. (fTT]h but makes an impact 
when active during a transition period. The system has 
the possibility to figure out whenever the later has hap- 
pened. When the input signal rf xt ^ is relevant then 



(n > o) 



and 



(r. 



(+) 



„(-) 



<0). 



(12) 



In this case the internal contribution + r\~ ) to the 
growth rate is negative and the input makes a qualita- 
tive difference. We may therefore define a global novelty 
signal S = S(t) obeying 



,(-) 



S 



vn 



(n > 0) and (n < r^) 
otherwise 



where we have used Eq. (|3|), r- + ^ -f r-~ 



-) 



n (ext) 



ri-r- ,and 
where a J2 i is implicit on the RHS of the equation. The 
novelty signal needs to be activated quickly, with T<t ^> 
Tg . Learning then takes place whenever the novelty 
signal S exceeds a certain threshold. 



4.2 Afferent link plasticity: optimization principle 



(1,2,3) 



(4,5,6) 



(0.1) 



(3,6) 



(1,2,4,5) 




Ar. 



Fig. 7 The activity levels xi of a dHan layer containing seven 
neurons, compare Fig. [6] the growth rates ri and the contribu- 
tions from the input-layer Ar%, see Eq. The first input stim- 
ulus does not lead to a deviation of the transient state dynamics 
of the dHan layer. The second stimulus modulates the ongoing 
transient state dynamics, influencing the neural competition dur- 
ing the sensitive phase. 




Fig. 8 Top: The 20-site regular dHan net used for the bars prob- 
lem. It contains 10 cliques with four vertices. The clique (0,1,2,11) 
is highlighted. Bottom: Some illustrative input patterns for the 
5x5 bars problem. 



Having determined when learning takes place, we have 
now to formulate the rules governing how learning mod- 
ifies the links afferent to the dHan layer. For this pur- 
pose we will use the Hebbian principle, that positive 
interneural correlations are enforced and negative cor- 
relations weakened. Our system is however continuously 
active, at no point are activities or synaptic strengths 
reset. The Hebbian principle therefore needs to be im- 
plemented as an optimization process [63], and not as 
a maximization process, which would lead to a poten- 
tially hazardous runaway growth of synaptic strengths. 

There are four quadrants in the 2x2 Hebbian learn- 
ing matrix, corresponding to active / inactive pre- and 
post-synaptic neurons, out of which we use the follow- 
ing three optimization rules: 



(a) The sum over active afferent links leading to 
active dHan neurons is optimized to a large but finite 
value r^ ct , 



(b) The sum over inactive afferent links leading to 
active dHan neurons is optimized to a small value s° rth . 



3 



n orth 



Xi active 



(c) The sum over active afferent links leading to 
inactive dHan neurons is optimized to a small value 
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The 



r act ^ r ina anc j g orth are ^ target values for 



a?i inactive 



the respective optimization processes, where the su- 
perscripts stand for 'active', 'inactive' and 'orthogonal' 
[64J. These three optimization rules correspond to fan- 
in normalizations of the afferent synapses. Positive cor- 
relations are build up whenever r% ct dominates in mag- 
nitude, and orthogonalization of the receptive fields to 
other stimuli is supported by s° rth . A small but non- 
vanishing value for r* na helps to generate a certain, 
effective, fan-out normalization, avoiding the uncon- 
trolled downscaling of temporarily not needed synapses. 



4.3 The bars problem 

Knowledge about the environment lies at the basis of 
all cognition, before any meaningful action can be taken 
by a cognitive system. For simple organisms this knowl- 
edge is implicitly encoded in the genes, but in general 
a cognitive system needs to extract this information 
autonomously from the sensory data input stream, via 
unsupervised online learning. This task includes signal 
separation and features extraction, the identification of 
recurrently appearing patterns, i.e. of objects, in the 
background of fluctuation and of combinations of dis- 
tinct and noisy patterns. For the case of linear signal 
superposition this problem is addressed by the indepen- 
dent component analysis [68] and blind source separa- 
tion [69], which seeks to find distinct representations 
of statistically independent input patterns. In order to 
examine how our system of an input layer coupled to 
a dHan layer, as illustrated in Fig. [6l analyzes the in- 
coming environmental signals, we have selected the bars 
problem [TOUT!]. 

The bars problem constitutes a standard non-linear 
reference task for feature extraction via a non-linear in- 
dependent component analysis for an L x L input layer. 
The basic patterns are the L vertical and L horizontal 
bars and the individual input patterns are made up of a 
non-linear superposition of the 2L basic bars, contain- 
ing any of them with a certain probability p, typically 
p w 0.1, as illustrated in Fig. [8j 

Our full system then consist of the dHan layer, which 
is continuously active, and an input layer coding the in- 
put patterns consisting of randomly superimposed black 
/ white bars. For the dHan network we have taken 
a regular 20-site ring, containing a total of 10 cliques 
C a , a = 1..10, each clique having S(C a ) = 4 sites, as 
illustrated in Fig. [51 The self-sustained transient-state 



process is continuously active in the dHan layer, modu- 
lated by the contributions {/\r^} it receives via the links 
Vij from the input layer. For the simulation a few thou- 
sands of input patterns were presented to the system 

M- 

In Fig. |S] we present for the 5x5 bars problem the 
simulation results for the susceptibility 



R(a,(3) 



Vi - 



a=l,..,10 , . 
/?=!,. .,10 [U) 



of the 10 cliques C a in the dHan layer to the 10 ba- 
sic patterns {y?,j = 1, ..,25}, the 10 individual hori- 
zontal and vertical bars, with C\ — (0,1,2,11), C2 = 
(2, 3, 4, 13), and so on. All cliques have the size S(C a ) = 
4 and the notation i G C a denotes the set of all sites 
defining the clique C a . At the start all are drawn 
randomly. 

The result is quite remarkable. At the beginning 
of the simulation the system undergoes an associative 
thought process without semantic content. During the 
course of the simulation, via the competitive novelty 
learning scheme, the individual attractor relics of the 
transient state dynamics, the cliques of the dHan layer, 
acquire a semantic connotation, having developed pro- 
nounced susceptibilities to statistically distinct objects 
in the sensory data input stream. This can be seen di- 
rectly inspecting the clique receptive fields 

F(<*J) = 7v7TT$Z^ a=l,.-,10, (14) 



S(C a ) 



iec a 



of the a = 1, 10 cliques in the dHan layer with re- 
spect to the j = 1, ...,25 input neurons, which are also 
presented in Fig. The clique receptive fields F(a,j) 
correspond to the averaged receptive fields of their con- 
stituent neurons. The data presented in Fig. [9] are for 
the 5x5 bars problem. We note that simulation for 
larger systems can be performed as well, with similar 
results [65] . 

The learning scheme employed here is based on opti- 
mization and not on maximization, as stressed in Sect. 
4.21 The clique receptive fields, shown in Fig. are 
therefore not of black / white type, but differentiated. 
Synaptic modifications are turned progressively off when 
sufficient signal separation has been achieved. This be- 
havior is consistent with the 'learning by error' paradigm 
[72] , which states that a cognitive system learns mostly 
when making errors and not when performing well. 

We may take a look at the results presented in Fig. [9] 
from a somewhat larger perspective. The neural activity 
of newborn animals consists of instinct-like reflexes and 
homeostatic regulation of bodily functions. The pro- 
cessing of the sensory signals has not yet any semantic 
content and internal neural activity states do not corre- 
spond yet to environmental features like shapes, colors 
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Fig. 9 For the 5x5 bars problem the response (see Eq. (1131) ), of the 10 cliques Ci, .., Cio in the dHan layer. The clique response is 
given with respect to the ten reference patterns, viz the 5 horizontal bars and the 5 vertical bars of the 5x5 input field shown below 
the graph. In the top row the cliques with the maximal response to the respective reference patterns is given. In the bottom row, 
below each of the 10 black / white reference patterns, the receptive fields, Eq. (|14|) . for the cliques C a with the maximal susceptibility 
(given in the top row) are shown color-coded, with black/blue/red/yellow coding synaptic strengths of increasing intensities. 



and objects. The neural activity can acquire semantic 
content, philosophical niceties apart, only through in- 
teraction with the environment. This is a demanding 
task, since the optical or acoustical sensory signals are 
normally overloaded with a multitude of overlapping 
primary objects. The animal therefore needs to sepa- 
rate these non-linearly superposed signals for the ac- 
quisition of primary knowledge about the environment 
and to map the independent signals, the environmental 
object to distinct neural activity patters. 

This very basic requirement is performed by the 
dHan architecture. The internal transient states have, 
at the start of the simulation, no relation to environ- 
mental objects and are therfore void of semantic con- 
tent. In the simulation presented here, there are 10 pri- 
mary environmental objects, the 5 horizontal and ver- 
tical bars of the 5x5 bars problem. In the setting used 
these 10 objects are independent and statistically un- 
corrected. During the course of the unsupervised and 
online learning process, the receptive fields of the tran- 
siently stable neural states, the cliques in the dHan 



layer, acquire distinct susceptibilities not to arbitrary 
superpositions of the primary objects but to the indi- 
vidual primary bars themselves. A sensory signal con- 
sisting of the non-linear superposition of two or more 
bars will therefore lead, in general, to the activation of 
one of the corresponding cliques. To be concrete, com- 
paring Fig. [9j an input signal containing both the top- 
most and the bottom-most horizontal bar would acti- 
vate either the clique C2 or the clique C$. These two 
cliques will enter the competition for the next winning 
coalition whenever the input is not too weak and when 
it overlapps with a sensitive period. The present state 
together with its dynamical attention field [14J will then 
determine the outcome of this competitions and one of 
the two objects present in this input signal is then re- 
cognized. 

5 CONCLUSIONS AND DISCUSSION 

The vast majority of neural nets considered to date for 
either research purposes, or for applications, are gener- 
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alized stimulus-response networks [751 174] . One has typ- 
ically an input signal and an output result, as, e.g., in 
speech recognition. In most settings the network is reset 
to a predefined default state after a given task is com- 
pleted, and before the next input signal is provided. 
This approach is highly successful, in many instances, 
but it is clearly not the way the brain works on higher 
levels. It is therefore important to examine a range 
of paradigmal formulations for the non-trivial eigendy- 
namics of cognitive systems, evaluating their character- 
istics and computational capabilities. 

As an example for a concept situated somewhere in 
between a pure stimulus response net and systems with 
a fully developed eigendynamics, we have discussed in 
Sect. 11.21 the notion of reservoir computing. For reser- 
voir networks the dynamics is, in general, still induced 
by the input signal and decays slowly in the absence of 
any input. Any given stimulus encounters however an 
already active reservoir net, with the current reservoir 
activity caused by the preceding stimuli. The response 
of the network therefore depends on the full history of 
input signals and time prediction tasks constitute con- 
sequently the standard applications scenaria for reser- 
voir computing. 

A somewhat traditional view, often presumed im- 
plicitly, is that the eigendynamics of the brain results 
from the recurrent interlinking of specialized individual 
cognitive modules. This viewpoint would imply, that at- 
tempts to model the autonomous brain dynamics can 
be considered only after a thorough understanding of 
the individual constituent modules has been achieved. 
Here we have examined an alternative route, consider- 
ing it to be important to examine the mutual benefits 
and computational capabilities of a range of theory pro- 
posals for the overall organization of the eigendynamics. 

In Sect. 12.11 we have examined a first proposal for 
the organization of the eigendynamics in terms of saddle 
point networks. In this framework the internal neural 
dynamics is guided by heteroclines in a process denoted 
winnerless competition. This neural architecture aims 
to model reproducible cognitive behavior and a single 
robust attractor in terms of a heteroclinic channel con- 
stitutes the eigendynamics in the absence of sensory 
inputs. 

In Sect. 12.21 we have examined the viewpoint that a 
non-trivial associative thought process constitutes the 
autonomous dynamics in the absence of sensory in- 
put. For any finite (and isolated) network these thought 
processes turn eventually into limiting cycles of tran- 
sient states. In this architecture there is however not a 
unique limiting cycle, but many possible and overlap- 
ping thought processes, every one having its respective 
basin of attractions. The transient state dynamics re- 



quired for this approach is obtained by coupling an at- 
tractor network to slow variables, with the neural time 
evolution slowing down near the such obtained attrac- 
tor relics. This is a quite general procedure and a wide 
range of concrete implementations are feasible for this 
concept. 

The coupling of neural nets having a non-trivial 
eigendynamics to the sensory input is clearly a cen- 
tral issue, which we have discussed in depth in Sect. [H 
for the case of networks with transient state dynamics 
based on attractor ruins, emphasizing two functional 
principles in this context: 

(a) - The internal transient state dynamics is based 
intrinsically on the notion of competitive neural dynam- 
ics. It is therefore consistent to assume that the sensory 
input contributes to this neural competition, modulat- 
ing the already ongoing internal neural competition. 
The sensory input would therefore have a modulating 
and not a forcing influence. The sensory signals would 
in particular not deactivate a currently stable winning 
coalition, influencing however the transition from one 
transiently stable state to the subsequent winning coali- 
tion. 

(b) - The eigendynamics of the cognitive system 
and of the sensory signals resulting from environmental 
activities are, a priori, unrelated dynamically. Correla- 
tions between these two dynamically independent pro- 
cesses should therefore be built up only when a modula- 
tion of the internal neural activity through the sensory 
signal has actually occurred. This modulation of the 
eigendynamics by the input data stream should then 
generate an internal reinforcement signal, which corre- 
sponds to a novelty signal, as the deviation of the in- 
ternal thought process by the input is equivalent, from 
the perspective of the cognitive system, to something 
unexpected happening. 

We have shown, that these two principles can be 
implemented in a straightforward manner, resulting in 
what one could call an 'emergent cognitive capability'. 
The system performs, under the influence of the above 
two general operating guidelines, autonomously a non- 
linear independent component analysis. Statistically in- 
dependent object in the sensory data input stream are 
mapped during the life time of the cognitive system to 
the attractor relics of the transient state network. The 
internal associative thought process acquires thus se- 
mantic content, with the time series of transient states, 
the attractor ruins, now corresponding to objects in the 
environment. 

We believe that these results are encouraging and 
that the field of cognitive computation with autono- 
mously active neural nets is an emerging field of grow- 
ing importance. It will be important to study alterna- 
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tive guiding principles for the neural eigendynamics, 
for the coupling of the internal autonomous dynamics 
to sensory signals and for the decision making process 
leading to motor output. Architectures built up of inter- 
connected modules of autonomously active neural nets 
may in the end open a pathway towards the develop- 
ment of evolving cognitive systems. 
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